Upgrade onnx runtime#1764
Conversation
jpggvilaca
requested review from
ActiveChooN,
MarkRedeman,
camiloHimura and
pplaskie
as code owners
jpggvilaca
requested review from
dwesolow,
jcchr,
mgumowsk,
piotrgrubicki and
romanowska
as code owners
There was a problem hiding this comment.
Pull request overview
Upgrades the @geti/smart-tools web ONNX Runtime dependency to a newer onnxruntime-web version to enable the webgpu execution provider, and updates the Segment Anything pipeline/session setup to work with the new runtime behavior (including worker-safe tensor passing).
Changes:
- Bump
onnxruntime-webto~1.24.3(and update the rootpackage-lock.jsonaccordingly). - Update WASM/runtime configuration and default execution providers to prefer
webgpu, plus serializeInferenceSession.run()calls. - Make Segment Anything encoder output structured-clone safe by returning a plain-object tensor representation and reconstructing the tensor in the decoder.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
web_ui/packages/smart-tools/src/utils/wasm-utils.ts |
Adjusts session defaults (WebGPU+CPU) and switches to a new WASM asset path/type. |
web_ui/packages/smart-tools/src/segment-anything/session.ts |
Updates ORT env/session creation usage and adds run-call serialization to avoid concurrent run() on a single session. |
web_ui/packages/smart-tools/src/segment-anything/segment-anything-encoder.ts |
Converts encoder output tensor into a serializable representation suitable for worker transfers. |
web_ui/packages/smart-tools/src/segment-anything/segment-anything-decoder.ts |
Reconstructs an ort.Tensor from the serialized encoder output for decoder execution. |
web_ui/packages/smart-tools/package.json |
Bumps onnxruntime-web dependency version. |
web_ui/package-lock.json |
Locks updated onnxruntime-web/onnxruntime-common and transitive dependency versions. |
Files not reviewed (1)
- web_ui/package-lock.json: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
jpggvilaca
force-pushed
the
jvilaca/upgrade-onnx-runtime
branch
from
44b8836 to
2ddb683
Compare
| // array data so the decoder can reconstruct a valid tensor. | ||
| const encoderResult: SerializableTensor = { | ||
| data: (await gpuTensor.getData()) as Float32Array, | ||
| dims: [...gpuTensor.dims], |
There was a problem hiding this comment.
can it be gpuTensor.dims?
There was a problem hiding this comment.
It's better to clone things, but we could maybe make them read-only. In any case i will update on my next PR
| return await this.ortSession.run(input); | ||
| // onnxruntime-web does not support concurrent run() calls on the same session. | ||
| // Serialize calls through a void queue so the result type stays clean. | ||
| const runNext = this.runQueue.then(() => this.ortSession!.run(input)); |
There was a problem hiding this comment.
could we make sure we have ortSession, i.e. not use !?
There was a problem hiding this comment.
100%, agree. Let me update on the next PR
| return await this.ortSession.run(input); | ||
| // onnxruntime-web does not support concurrent run() calls on the same session. | ||
| // Serialize calls through a void queue so the result type stays clean. | ||
| const runNext = this.runQueue.then(() => this.ortSession!.run(input)); |
There was a problem hiding this comment.
I don't really understand why we need this, could u please explain more?
There was a problem hiding this comment.
Artifacts from Claude, but the gist is that each run() waits for the previous one before starting, so results always resolve in call order.
In any case i agree this is confusing so i will refactor this as well on my next PR. It will be something like:
const next = this.runQueue.catch(() => {}).then(() => session.run(input));
There was a problem hiding this comment.
Maybe we should cancel ongoing processing if there is a new one?
There was a problem hiding this comment.
We want to wait for it. The user clicked so it's intentional. Worry not. The next version will be much simpler :)
📝 Description
webgpu.Manual tests:
Before:
After
Next steps will be: Update opencv to the latest version, and lastly, update our old Sam1 model to Sam2.1
✨ Type of Change
Select the type of change your PR introduces:
🧪 Testing Scenarios
Describe how the changes were tested and how reviewers can test them too:
✅ Checklist
Before submitting the PR, ensure the following: